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ABSTRACT ' • 

Project Follow Through has not yet been properly 
evaluated by researchers guidled by appropriate a.ssumptions . Tests 
use^ in the basic Follow Through evaluation had no clear relationship 
to most Follow Through programs. Still, conclusions of the evaluation 
have been presented with certainty and lead to the judgment that ^ 
Follow Through does not work. Suggesting otherwise is the fact that, 
at some Follow Through sites, betweeh 10 and 13 percent of the ' 
children have scored above the 50th percentile on the Metropolitan 
Achievement Test. This fact should be examined ii^ detail through 
comparisons of high- and low-^perf orming programs. Thete is fauch to 
recommend such a comparison. Certainly, it is not di£fici\i3t to .find 
educational data that reveal lon^-lasting, positive outcomes of good 
teaching for students thought to have limited potential for 
achievement. Along with a focus on child outcomes, future Follow 
Through, evaluations should focus on teacher k^ehavior and the way. 
treatments, or models, ftre impl'smented. It will be absolutely 
necessary to insure that measurement ijostruments employed *are 
appropriate to the task. The methodology for making prepise.^ 
comparisons among educational treatmehts presently exists. Contrary 
to generally held views of experts. Follow Through has not been ^ 
failure; Follow Through children have not failed to respond to good* 
teaching. When educators become clear about the goals of educfation 
and commit themselves and their resources fully to good teaching, 
academic success for children will be assured. The alternative is a 
dangerous academic rationalization and legitimization ot the status 
quo. (RH) 
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Large-scale educational evaluation ..studies have tended to conceal much 

♦ 

mojre than they have revealed. This is particularly true of .the major national, 
studies on "program effects". Many of these studies, such as the VJestinghouse 
Head Start Study (1969), the.Col«man Study of Secondary Schools (1966), and the 

ft 

Abt Day:.Care Study (1977) are w^l known for their findings of small or no slg- 

• * * - 

nlflcant program effecCs. These repeated findings hove been accompanied by 
deep-seated feelings of despair among professionals and parents who *have de- 
voted trelhendous amounts of emergy toward the improvement of the quality of 
education fir young children. However » their despair can in no way equal that 
•which is found in communities wl^ich* are sferVed by such pro;^rams where the cit- 
izens have hoped for so much. If we take the results'^ of the primary national 
studies of educa onal innovations at a national level in general^^we might 
feel compelled to conclude t^l^a^ everything which can be tried has been tried 
and that, as some evaluators have suggested, we should lower our expectations, 
both for finding effective programs and for ralsl'-j^ the achievement of chll- 
dren* If we are to believe the generals evaluation results in education, then 

we must conclude that the children of the poor are doomed to an eternity of 

r 

educational failure, • - \^ 

4 * 

One exception to the type of results whic\i are generally reported in 
major national studies of all types and at all levels of education is the work 
of Ron Edmonds . (1979) and others which has attempted to study effective 
schools. The Edmonds, Approach to educational evaluation^ or research, dep^d- 
ing on your point of viev», represents a radical and profound shift in the type 
of auestions which the evaluator askg • The typical questi6n ixx national eval- 
uations of^federal education intervention it'programs*' is : Do all schools, pso- 
g^ams, classes, i^.tc, on the av erage , work? Edmonds has turned the Ic'vge scale 



evaluation approach upside down. His question is at once both simple and 

profound* It is not a question of .^'dbes a national ^program' work?" but *^do 

any school sites , ""and then, by extension^ does any program work?" . His some- ^ 

what star^tllng finding Is that there are Indeed many "garden variety" schools 

that serve ordinary children, ordinary parents, in ordinary low-lncomc neigh- • 

borhoods, 'which do succeed in getting hi^gh quality academic results for chil- 

dren comparable to those which kLght be expected in other places, as far as 

the basic skills ^re concerned. In typical program evaluation^, the nature 

of tf.he aggregation of data has caused such instances af schpol suocess to be • 
• *• 

overlooked or submerged in the basic data* Edmonds* work was quite different. 

lie took hla analysis one step further. He chose to do'a more fine-grained 

' • .■ ' • " ° ■ • 

analysis of those schools which were successful In comparison to those 

schools which were quite unsuccessful^ and was able to idefit^fy certain typi- 

cal characteristics, for successful schools* Then he took an additional ^ ,and^ ^ 

most courageous, step, and went to work in the New York City Public School; 

• ■ 

System to demonstrate that it would be possible, using data from his school 
effectiveness studies, to interVene in schot)ls to Improve success* The pre- 
liminary data from his Mew York City work indicates a significant degree of 
success in this' regard* , 

To some extent, the Follow-Through evaluation is^ comparable to the Ed- 

"X > * 

monds approach* On one level, it asks an Edmonds-type question: ''Do any of 

thei 'models' work?" However, at another level it repeats the errors of pre- * 

vlous glqbal studies of national education interventions, or of regular school 

operations* If- the follow-Through study had asked, "Do any Follow-Through 

sites work?" it undoubtedly would have located manj^ such sites, as is sug- 

gested in parts of the Abt Report* However, by treating the "models" as if 

they represented unique and consistent treatments, it might well be expected. 

that successful Follow-Through operations would be submerged in the aggre- 

ERIC '4 . 
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gated data, since few models are artldulated precisely or guaranteed to have 
been Implemented* Kote will be said about this later on« 

, When those who were responsible for making decisions about the develoi^- ^ 
ment of a FblloV-Thrpugh thrust were operating in the initial stage » at least « 
two kinds of assumptions could have been made about; the reasons for the poor 
performance of poor African- American 'children and other chfldren* served by the 
program: developers might have reasoned that^he children* a low achievement 
was due to ,the fact thafea they had beeu deprived of adequate instruction inf 
school; or developers couldf have concluded that the ptior performance of Fotlow- ^ 
Through children was due to oie fact that the children or their families 
pdsse^sed some deficit which required unique iiitfervention strategies.' 

It makes all the difference in the world which of the^ tifo hypotheses 

* • ■ 

desi^gners happen to favor.,' If the former is accepted, then the Follow- Through 

program logically would simply be a matter of providing sufficient resources 

\ ^ 
to parents in order that they might provide the san^e typtf of instruction which 

is available to the families which. are more affluent than they. On the other 
hand, if the»-second hypothesis is accepted, then the Follow-Through program 
becomes a kind of experiment to test the! validity of various unique ''alterna- 
tive" strategies which w(>uld solve the "puzzle" of how to teach the iCljiidren 
who are setved by Follow-Through. It appears that supporters and antagonists 

alike of Follow-Through programs are quite together; .f^or t\^e most ^part, in 

* * 

emphasizing the second assumption. It certainly appears to be reflected in 
the general character of^ the Follow-Through operation and in jLhe^character 
of the formal evaluations. For e^^ample, one might ask the hypothetical ques- 
tion, "if on the average, a parent of a Follo\ ^-Through child were provided 
with the resources" to ?end their child to a high quality expensive private 
preschool and primary school program, would we expect there to be. significant 
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positive differences for the child in the new setting versus the old?" To an- 
*i3wer "yes" Is to 'cdme down on the side o£ Follow-Through as a resource program^ 
since program quality is knowable and determines 'the quality of a^ child's aca- 
demic progress. To answer^ "no" is to come down on the side of Folluw-Thro* gh 
as a problem-solving program, since there is a presumption that we do not know 

how to teach children who are poor. It should be remembered that the oasis for 

* * 

the observations of Follow-Through programs in evaluation seem to derive from 

the second assumption. . " 

The Validity Df Evaluation Ing'truments 

It is important that brief mention be made of the actual instruments which 
were used in the basic Follow-Through evaluation. Among those considered were 
^he Wide-Range Achievement Test, Preschool Inventory, the Metropolitan Achieve- 
ment Test, the Ravens Colored Progtess^L-AXe Matrices,, the Coopersraith Self-Esteem 
Inventory, and a Questionnaire on Intellectual Achievement Responsibility. The 

* * • 

four instruments whic^ figured prominently in the final, evaluation were the 

l|etrop6litan Achievement Test, the Ravens Colored Progressive Matrices, the^ 

Coopersmith Self-Esteem Inventory, and the Intellectual Achievement Responsi- 

bilitjT Questionnaire. To anyorie Who is familia.r with these instruments, it 

should be clear that their relationship to most Follow-Through programs is min- 

imal to nonexistent. In this author's experience, ,few educators who use the 

results of measuring instruments are familiar with their contents. Content val- 

» 

idity for, measuring instruments is a basic professional requirement. For edu- 
cators' familiar both with Follow-Through programs and with the four instruments 
se\e\u:fid as criteria for use in program evaluation, the mismatch between pro- 
grams and instruments for their evaluation should be crystal-clear. Of all^the 

Follow-Through programs that I have visited, I have seen none which used Ravens 
\ 

Matrices as program content. Some psychologists and educators appear to support 



the Idea that the Ravens Colored Progressive Matrices Test Is really a test of 
higher cognitive abilities rather than, a simple achievement test. And yet| 
the work of Reuven Feuerstein In Israel has shown clearly that learners can 

* 

be taught to solve' Ravens type problems quite easily. 

This leaves the Metropolitan Achievement as the only other cest of aca- 

demic achlevemerit. Once again, it becomes critically importfant to determine 

I 

empirically if the content of the Metropolitan Achievement test matchesj the 
content of Follow-Through programs. In the absence of such empirical demon-^ 
stratiori, the use of the Metropolitan Achievement , Test as a universal criter- 
ion for Follow^Through evaluation must be regarded as highly suspect. 

In my opinion,^the Coopersmith Self-Esteem InyentoryM.s Inappropriate for 
Follow--Throush evaluation. Any program evaluation is dependent upon the use^ 
of valid instrumentation. And yet there aria not sufficient yata on the use of 
the Coopersmith Self Esteem Inventory with the diverse cultural. groups repre*- 
sented in Follow-Through programs at the Follow-Through' age to meet even the 
most minimal professional requirements. Little need be said at all about data 
in support of the Intellectual Acjiievement Responsibility Questionnaire. The 
basic point is this^ without the existence of valid instrumentation, the pro- 
gram evaluation is fatally flawed. And yet v^hen we come to final evaluation 
of the Follow-Through program, we find little admonitions for caution in the 
interpretation of the data. 

It is hard to find a villain in -t^iis^^tuation. Pressures arc enormous 
on all parties to the Follow-Through experience. Any contractor who a^^ees 
to do an evaluation will be under extreme pressure to get the job done without 
excuses and with *'the best instrument that we have at the time." Federal 
agency personnel likewise are under tremendous pressure to deliver to the 
Congress and to others definitive answers to t?he question "Does Follow-Through 
work?" Public policy makers also experience tremendous pressure to be account- 



able, for pubFllc funds and 60 demonstrate to constltuejits thati^^e^ectl action 

• r ' . , ' * ■ ■ ^ 

Is belong or has been made, on their t>ehalf. The conb:f '^tlon of these and other 

pressures on these and other .artles^to the Fol^low*^" irocess forces aval-- 

uators to take short cuts. It Is difficult -If not Impwsslble, to avoid the ex- 

.pedlent solution. And yet It can have dire consequences. 

The FollowThrough .Results . , / " « 

The Follow- Through evaliiation^^has offer. * the following conclusions. An- 

ders^n, Pierre, Proper and Stebblns (1979) Indicated that the following were 

their main findings: . f , . * 

1. "Each Follow-Through model had very different effects on^test scores 

ift the various communities in whigh it was Impremented. Differences in*^ 

effectiveness between sites within each model were gi^eater than overall 

differences in effectiveness between models. .None of the 17 models »in " 

ft 

, the evaluation demonstrated that it could compensate conc^istently for the 
academic consequences of poverty. From this finding^ . . • with whljch thte 
House group agrees , we coriclude that the Follpw- Through strategy of ex- 
ternally sponsored cvirricular change is not a reliable tool for raising . 
'the test scores of poor children; th^e strategy is sensitive to too'taany 
influences over which it hjas no control.- Local circumstances and behav- 
ior clearly have more to do with children's^ test performance th^in do the 
intentions, theories, and rhetoric of outside interveners." ^ 

^ 2. "In most cases the Follow-Through group scored about as one would ex- 
pect similarly disadvantaged groups to score without Follow-Through. 
Where differences were apparent. Follow- Through groups scored lower more 
frequently than they scored higher. ... it appear'! clear, then, that 
the Follow- Through strategy is not an effective tool for raising poor 
children's test scores. Nolf only are the effects unstable, but they are 
^all on the average, and a :disquietingly large minority of them are in 
the wrong direction.'* \ ' 



3. "With few exceptions> Follow-Through groups were still' scoring sub- 
stantlally below grade level at the end three or four years* inter*- 
ventlon • • • • poor children till tend to perform poorly Un school, 
even After the best and^the biU-ghtest theorists — with the help of parents 
local* educators, and federal funds » and supported by the full range of 

supplementary services associated wllih community action programs^^ave 

- • ■* * ' ■ * 

done their best to change the i^ltuation." 

If the pessimistic conclusions which ar'e presented above are taken at , 

' * . •• ■ ' • * 

face value» the likely public policy consequences sho\]p.d be ap^parent to anyone 
Theibaslc point of these conclusions Is .that ^'Follow-Through does not work.'^ 

... • Z' , 

The unfortunate thing about these conclusions Is that they are presented with 
syoh a ring of certainty — certainty th^ the best efforts of a number of * 
peopre have been given and that they have failed. lAnd yet, what was t^e "Fol- 
low- Through program"? Was the program In all its varied forms, faithfully 
Implemente d? Were the evaluative crlterl^^ongruent with program goals ? Did 
valid instrumentation exist to determine Che extent to which the goals were 
achieved?' To leave the ma^er as it stands is to level an implicit indictment 
of the children of Fallow-Through and/or their families as being, deficient in 
some w^i^^ And yet there are' at least two other sources of explanations for 
the apparent lack of progress among Follow-Through children. One the one 
hand, progress may have been achieved and yet not revealed because of invalid 
evaluation. On the cftiher hand, progi;ebs may not jiave been achieved due to* the 
quality of the educational treatment which Follow-Through children actually 
received. Up to now, a good deal of attention has been focused on the matter 
of tlie adequacy -of the evaluation. 'Less attention has been focused on the pre- 
cise nature and adequacy df the educational ^'treatment", which the children ^ 

actually received. Because of this, the evaluation of Follow-Through has not 

v. 

yet been accotnplisli/ed. • ♦ ^ . • 
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In "Perspectives W the Follow-Thrr ugh Evaluation " (1978), with the ec- 
ceptlon of the broad view taken by Walter Hodges, critics and supporters of^** 
the Follow-r Through evaluation study looked very narrowly -at matters of ex- 
perimental design and ins trument Validity . While^^diectie&ants alchibited tech- 

^ ' * * 

inical adequacy regarding the^ topics which they chose to review, to anyone who 

t. * . • ' 

is familiar with the operation of Follow-Through operations on a day-by-day 

basis, these- technical ^discussions would seem quite remote, not only because 

* 

of thedr technicality, but because of thei types of things which were, not dis- 
cussed but which are important in Follow^Through programming, Walter Hodges ; 
rrises .import^anC questions, such -as the matter of parent involvement, school 
Climate, and curriculum development. All of.-the&e were impottantJ process 
goals for Follow-Through, yet they received little, attention in the evalua- 

•tion effort. But most important , Hodges paint?^ to the finding'^ that at soTtya 

> . ■ • 

Follow-Through sites, between 10 and 13 > percent of the Follow- Through chil- 

dren were above the 50th percentile on the Metropolitan Achievement Test\ 
There is a tendency, in the kind of analysis which has .been perf or -^ed in the 
evaluation and in the kind of analysis which serves as a basis for the cri- 
tique, to treat such an unusual finding almost as if it were i random varia- 
tion rather^ than" as something which needs* to be examined in detail, \fhat was 
' the nature of the Follow-Through program at the sites where the children's 

achievement was above the national aver^^ge? High performing sites and low 

■ a 

performing sites should be isolated for comparative anaj[.ysis. 

Because of the structure of Follow-Through. historically , it woul'd be \ 
difficult at this point to do such an analysis. For example, it *is anything 
but clear that the Metropolitan Achievement Test reflected specific program 
goals for -Follow-Through sites, even for those sites^ that did well on the 
test. And yet the Follow-Through program is too importarft to too many people 



ERJC ■ . XO 



to allow such divergences in the trends to be submerged in national averages.. 
Surely those teachers , parents and children at the high performing sites would 
feel devastated to learn that their achievements were effettively disco^mted* 
Afid yet these sit^s do not stand out in any of the material^ on Follow-Through 
evaluation or critique • * * ' * , 

r 

A look at related experience ^ ' And so', given the data from fo*:r^i evaiua- 

« 

r 

tions, one could say that the great Fqllbw-Throvgh experiment seems, to have 

taught us very little. This is not the same thing as saying that there were 

^ •• . ■ . • ' ^ 

lessons to be learned. Perhaps a .different look at FollQW-Through and alterna- 

tives with whidh it may compared will suggest ^w ways of looking at this and 

at other such social programs in the fut;ure. It it hard to believe that anyone 

would question serio^usly the "value of a quality education during the early years 

for young children which would be. followed by attempts to sustain that level of 

quality instruction over additional year^ In the^minds^pf ail parents who' have 

the financial resources to cover thd costs of education, there is no question 

but that education "works." We are. left then with the question of whether Fol- 

low-Through is education, whether education works only for some children, or 

whether affluent parents are mistaken a^ter all. In other words, is it less 

a question of whether the program works than whether som^ poor children can 

do the "work*'? Perhaps there are still those among us who 'believe tthat chil- 

dren in poverty are so damaged by their conditions that they are unable to ^ 

learn. On neither of these two questions does the formal Follow-Through 

evaluation provide us with adequate answers. And yet we may appeal to other 

data, should such an appeal be necessary. These data will show unequivocally 

that quality educational programs do '"work" and that quality education works 

for virtually all children. If these things can be shown in other settings, 

then We have a new question; Why can't they be shown in the Follow-Through 
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formal evaluation data? It /Should be clear at onca that by limiting the 



l'9llow-Through evaluatJUjli to a look at the Follow-Through program and to 

the Follow-Through population, it is highly likely that we have limited our 

ability to see important things about the education process in general and 

Follow- Through in - particular . 

xhere are many intervention 6r educational programs which achieve high- 

quality results with all types of children. Some of these are' Follow-Through 

sites 9 as we have learned from Hodges* citation. .The field of education Is 

full of examples of dramatic acliievement by students who were thought to be- 

long to racial, ethnic 'or economic groups that had a limited potential for 

education. * Some will be mentioned later. The power of teaching, to change 
» ^ 

the*^*predicted outcomes for such students has been demonstrated over and over 
again. The changes for students appear to be quite long-lastin|. If the 
attention of educational researchers had been focused more closely on these 
successful examples, I do not doubt that there would be more -abundant public- 
ly known data to suppart this assertion* Even with the absence of a suffic- 
lent quality of long-term systematic evalu^ition attention, we may cite sev- 
eral examples. One of the most popular programs' on 60 Minutes was a segment 

which featured West Side Prep High School, where Marva Collins is the Dir- 

X 

ector. The childreu who are served at West Side Prep are children Who live 

in one of the lowest socj.oeconomic areas in the city of Chicago. They are 
* 

alsp children who, in most cases, were performing very poorly in school 
prior to the time of their entry into West Side Prep. The results of Marva 
Collins* dedicated work are there for all to see. The children have been 
overwhelmingly 3ucc;essful in precisely those areas of curriculum which were 
thought by many to be beyond their reach: mathematics, science, and, especi- 
cilly, English and literature. 
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A brief visit to the Oakland Community School in Oakland, California, 

reveals a school with similar results. Located In one of the worst poverty 

pockets In Oakland > It hae shown chll'dren from, for the most part, very low- 

1 . . ■ . 

income families to be quite capable of mastering regular academic school 

1 

subjects at the highest level. As in the case of West Side Prep, the chil- 
dren who have been helped are those who previously had^ had major difficuiies 
in regular school environments. The same may be said for the Marcus Garvey 
School in Los Angeles, the Nairobi Day School in East Palo Alto, California, 
the McKinley Educational Institute in Berkeley, California, and a variety of 
other such schools scattered throughout the nation. W^have already mentioned 
the work of Ronald Edmonds, which is important becatl^e his attention hUs been 
directed nationally toward ordinc ' "garden-variety" public schools which, 
aga ist all odds, are succeeding be.uutif ully . One wonders just how many suc- 
cesses are required befoire-^it^ is understood clearly that the quality of 

4 

teaching still explains much more about pup^l^perf ormance than does the socio-* 
economic background of the students % ^-^^^ 

Our evidence for this is not limited to those few examples- cited above, 
or even to the large-scale national study' by Ron Edmonds. Reuven leuerstein 
( 1979) has shown clearly that "thinking skills" can be taught in a relative^ 
ly brief period of time to thousands of "retarded" children who may have been 
thought to be uneducable beyond a minimal level. Renee Fuller (1977) has de- 
monstrated the ease with which reading can be taught to "retarded" populations 
whose scores on IQ tests would indicate that they would be incapable of sig- 
nificant academic achievement. Paulo Freire ( 1973) has also shown that even 
as late as adulthood it is possible to teach reading in approximately 30 
hoyrs' time ^o adults who have spent all their lives in abject poverty. The 
list goes on and on. For those who would doubt the existence of such schools 
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and classrooms » ^or who would doubt the power of teaching to produce with 
comparative ease significantly greater levels of pu()ll achievement than we 
normally find in poverty areas, it is unlikely that extended citation of such 
examples of teaching power would serve to change those opinions. However, 
for those who do believe this evidence- and who may have had many similar ex- 
periences of their own to report, a fundamental question arises:. Why does the 
Follow-Through evaluation ^f ail to yield information of the type Which has 
Just been cited above? If Follow-Through programs which wer^ dramatically 
successful actually existed but Were not revealecTTn the evaluatipi^, then 
something is wrong with the evaluation. If, on the other hand, even the best 
Follow-Through programs actually yielded only minimal academic results, as- 
suming the criteria and instruments to be appropriate, then something must be 
wrong with Follow-Through, since we do have the evidence that children's ^aca- 
demic achievement can be chaxiged, sometimes with quite minimal resource,s« 

Some of the work in Head Start aad Follow- Through, the research work, 
has been done for the purpose of gathering data to inform public policy de- 
cisions (Abt, 1977 ) . Decisions about care-giver quaiif ications, group size 
class sise, and so forth, have been tied to information which is generated 
from a look at Head Start and Follow-Through programs. And yet, as has been 
shown, these national studies generally leave out the use of programs or 
schools \which are known to be highly successful, even within the Follow-* 
Through i^ffort. Consequently, by habitually, arbitrarily, or thoughtlessly 
restricting evaluaticr" attention to a comparison of Follow-Through "programs" 
a great opportunity has been losu. 

Of. course, this suggests some things for the future for Follow-Through 
research and evaluation. What precisely are the ingredients of the many 
successful early childhood programs and later childhood formal school exper- 

14 ■* 
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lences which account for success In certain '^Qlrcyinstances? If such research 

/ 

Is conducted^ one must be prepared for a fundamental challenge to some of 
our* most cherished notions* Caregiver qualifications In successful schools 
bear little reseoiblance to proposed or^ publicised caregiver qualifications 
which are thought by many professional* educators to be essentials - At pres- 
ent, public policy decisions. If based on data at all, are based oh the most 
llmifeed data pool — data- which come almost exclusively from federally spon-. 

sored/ programs. If Follow-Through evaluation gives us the opportunity to 

»^ • 

change th^ situation^ so that the wider arena in education is considered, 
a major contribution will have been made to evaluation and to the field of 
pedagogy. 

I for one remain unconvinced by the formal Follow-Through evaluation 
data that the true power of the Follow-Through effort has been minimal. I 
believe that v.he eva;Luatlon approach Itself is much weaker than the Follow- 
Through effort which it seeks to assess. Neither articulated criteria nor 
measuring instruments of sufficient sensitivity exist at the i)resent time to 
allow us to do valid assessment of Follow-Throug^h. Further, if all Follow- 
Through programs are expected to have the same goals and objectives, then 
that policy decision must be reflected in program design, staffing, imple- 
mentation, and supervision. For example, we know enough now to design edu- 
cational experiences for children which will cause almost all of tjitm "to 
meet the minimal academic competencies which schools require. In fact, for 
most if not all children, achievement should be x;ell beyond the minimum aca- 
demic requirements. While formal data from the Follow-Through experience 
may not have taught us ♦^his, the fact that it has not suggests a need for a 
major overhaul of evaluation, program or both. These are policy matters* 
We cannot have planned variation in programs and planned restriction in 



program outcomes. Either policy decision must be made to design the evalu- 
ation In advance of Follow-Through programs and to link program planning to 
the explicit goals, or the program evaluation should be eliminated alto- 
gether, as it is with education generally. - * 

It is my opinion that evaluation does not serve us well if it sets up a 
mental orientation that^alLows for the acceptance of conclusions from present 

Follow-Tbro^h evaluation data, along with the belief that "Everything that 

^ could' be done has been done." In the final analysis, Follow-Through chll-* 

* I. * 

ctren need one thing^'more than anything else; they need to be guaranteed a 

hJ.gh quality education. It is only after that quality has been. guaranteed 

9 

without the expected success in children's achievement that special "inter-* 

, » ». » ^ 

ventionsl* or research need .be conducted as a way of helping children. 

And so, in the case of Follow-Through, the Office of Education did not 
offer its support to programs of known quality, ask that they be implemented 
and that their results be "evaluated"; it asked essentially a set of research 
c^uestions which may be summed up as follows: "Can the problem of finding suit- 
able ways of educating the poor be solved?" "The hope , of many individuals in 
Washington was to find better ways to educate poor and ^Inprlty chij.dren" 
( Harvard Education Review , 1978, p. 125). To ask if an educational program 
"works" prior to the time that either the program or its objectives can be 
.articulated is to ask a research question which uses the language of evalua- 

# 9 

tion« Tol low-Through research data has been tre'ated as if it were eval^uative 
data. We should have learned by now to distinguish between these two things. 

• The failure to make this important distinction has major consequences 
for communication and for the mobilization of support for a so ial program 
among the various program constituents. It is important to have agreement on 
who is responsible for what and to know in advance just what a "program" can 
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or cannot do. If parents and^ children are participants In an experiment > then 

they should'' be so informed and the experiment must fall under the Informed oon^ 

*■ • * 

' i ' ^ ' ' ' 

sent guidelines for^ the protection of human subject^s. On the other hand^ 1^ 

^ •■ ^ 

the details of a non-experlftental program of known quality are described fully » 

then all parties to the program can and should be In a position to monitor ' 

r 

every asTpect of Its Implementation. Then evaluation Is possible, 

* 

Defining a Follow-Through mpdel v^ % close Inspection of the descriptions 

' ''■^ • ...... 

of Follow-Through- models will reveal that there are InY^uff iclent dat^ to deter-* 
mine with adequate precision just what the unique character of th^ educational ^ 
model was In virtually any txf" the case^. This certainly Is not to. say that 

N 

good education could not ^ave been conducted » or was not ' conducted i under the^ 
rubric wof a ''model.'' In fact/ In the judgment of many, good education did oc- 
cur at many Follow-Through sites. However, an e^tamlnatlon of descriptions of 
Follow-Through mpdels fall? to show that a paradigm exls^ for model defini- 
tion which would allow precision distinctions to be made among the variations 
for either research or evaluation purposes. 

' ^eep data on tfre implementation of a "model ^'. Ass\imlng that the parti- 
culars of a model d^uld be articulated according to some common way of talking 
about models, another step would have to be taken: that is, to determine if ^ 
the model which had been described for a given Follow-Through program was uni- 
formly imp^mented at all si^tes throughout the program. This point was made 
in the House, Glass, McCiean and Walker critique (1978). The failure to do 
this has resulted in the apparent presumption that the loosely described "mo- 
dels" were in fact uniformly and fully implemented, since there are few data 
in the technical reports which allow for a judgment to be made regarding the 
degree to which the stated model was Implemented. Indeed, it would be a maj9r 
evaluation effort simply to verify that implementation' of models. 
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^ Stalllngs* Inltiar probleiD was specification. Th^re were no 

comprehensive descriptions cff spohsorri' models in terms of class- 
room behavior. Without such descriptions it would be Impossible 
to assess how well models had been Implemented* So in t}}e spring 
of 1973, "Stalllngs Visited "ideal first and third grade classrooms 
specified by each si^onsor"'ln the hope of defining models in be*- 
havloral terms by observing ideal moviel classrooms. *'What would^ 
be desirable for a closely controlled experiment would ^e ^n expli- 
* clt statement by sponsors of what proportion of the time critical 
variables would ocj:ui^ in an Ideally implemented nilassroom. How- 
ever » elen\ents itf a classroom are not like those in attest tube. 
We have not yet learned to predict the amount of individual atten- 
tion or feedback that a group of children needs to meet specific - 
goals. Each classroom Ife made up of individvials, and individuals 
V are most likely to need different rates of feedback or individual- 

^ Ized instruction for maximum growth. Thus , even the mos^t specific ' 
models » such as the University of Oregon and the University of 
Kansas 9 would alter the rate of reinforcement depending upOn the 
needs of the child <^ ... We simply could not come up with a recom- 
mendation that would specify the rate of questions or reinforcement 
^to be expected from even the most structuired models.*^ In conse- 
quence^ the attefmpt to specify ideal model types directly did nof 
work.. (Tl ^ Follow-Through Report^ Vol* V, p. 157) 

The failure to define a modpl adequately and to guarantee its impl^^r* 
mentation in an evaluation study has placedP Foliow-througii children and , 
their families at great risk^ since, as data are reported: the public and 
casually informed policy makers assume that articulated programs were 

fully operational. Therefore the explanation for failures to find larger 

♦ 

significant differences between Follow-Through children and non-Follow- 
Through children tends to locate the blame for "program failure" with the 
children and/or their families. One could pred^jct with a fair degree of 

certainty that the next step to help Follow-Throygh children would in all 
likelihood not be attempts to look at ^models, but rather to devise new. 
"models" j^or the purpose of solving the continuing "puzzle" of how to edu- 
cate children who have been supported by Follow-Through. The question 
always seems to come out, "does education i work?" We never seem to be able 
to ask the question, "did education happen?" or "what type of schooling 
happened?" 

18 



..^ ■ ' It is worth noting again that amohg all the mounds of mater if 1» 
resulting from the Follow-Through evaluations^ these descriptions 
by Stallings are unique. They comprise the only direct empl^^al 
descriptions pf what life is like in the class roomw oif different 
sp'onsors. (The Follow-ighrough Report . Vol'. V, p. J.59) , 

One of the most lmpo:^tiaift lessons from the Follow-Through 'iprogram Is 

that the tfeajtments which children receiy^ vary . Perhaps an unintended con 

seqifence for the conduct of Follow-Through evaluations is to document the 
» «« 

variety of treatment which children receive both within and outside school 

or ra;:her9 treatments in special programs when compared to non-programs. 

Man}^ prof essio'nals appear to hold an image in their heads of teaching as a 

standard "treatment". . ^ 

All fiv^ of the Wjor Follow-Through studies — the Emerich and 
V^^^il^^e Abjt I, II, III, and IV Rjepoxts — discuss the , implementation issue 
and its importance to interpreting effects estimates, but none of 
^ tho.se Follow-Through evaluators, neither Emerich et al., nor Klein ^ 
and company, nor Anderson, Stebbins and associates, could make much 
progress on the problem. Their reactions to this frustration were 
similar — ^^they wanted 'more St^nformat ion. Emerich proclaimed the "need 
for more precise treatment definitions and descriptions" . . • K16in 
and company callfed for a "directory of effects associated with types, 
of sponsors under different types of conditions" . . ♦ and in their 
summary of the Abt IV Report, Stebbins and associates advised "future 
evaluations that aspire to explain the outcome of compensatory inter- 
vention will d9 well to investigate thoroughly the local circum- 
stances, attitudes and activities that clearly decide whether a com- 
pensatory education program dan work or not. (The Follow-Through 
fleport . Vol. V, p.'^'lS?) ^ 

• 

It has been only in recent years, through the work of people such as 

0 

Jane Stallings at' Stanford Research Institute, Fred McDonald at the Educa- 
tional Testing Service, Jeri Brophy at the University of Texas, Raymond 
Rist, and others, that the wide variations in treatment which children re- 
ceive have been described systematically. The -effect of these data should 
be to cut the ground from under those who choose to attribute to children 
or to the:^r familie^ the sole or the'major responsibility for chlldren*s 
school achievement or the lack of it. One of the^ardest things to do in 
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education has beett to focus professional 'attetitlon on what professionals do. 
Even In the. Follow-Through evaluation, when the data are provided on e<iuca- 
'tor behavior. It appears to ''be an afterthought comprising the smallest pari 
of the study; and while, in my opinion, it «should form a major and Indis- . 
pensable part of any interpretation, it is virtually abs^ent from the final 
summary report of «he evaluation. Even^then it occupies, too little of the. 
attention of the critics, Weikert and Benet Jtiave^summea" the matter up 
nicely: "It is not neceslSiariiy ttie methodology that is at fault, even in 
^quasi-experiments' like Follow-Through • • • what Is needed may be a clear- 
er conceptualization of models, outcomes, and intervening processes, as well 
as better educational treatment for the researcher to eValuate*' (The Follow- , 
Through Report s Vol. V, p. 164). 

Ins trumentatioii . The use oT an achievement test, the MAT , an IQ 
test, the Coloured Ravens Progressive Matrices , a self-concept test, the 
Coopersmith Self Esteem ^nventory > and a home-made ''locus of control"^ test 
as the criteria for whether a program works or not, is much more than a 
necessary compromise in order to -utilize the "best instruments available." 
It is a virtual surrender to triviality. There simply mu»t be some point 

beyond which essential meaning for evaluation has been compromised away. If 
existing standardized instruments allowed most of our evaluation quest^^ons ^ 
to be answered, one might say that a compromise which settled for incomplete ^ 
or inadequate instrumentation would be useful. On the other hand, when the 
mismatch between the content of instruments and program descriptions is as 
gross as appears to be the case In Follow- Through, especially with "planned 
variations", then a little bit of information may well be worse than none at 
all. For example, to use the Raven Progressive Matrices Test as the measure 
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J ox Follow-Through outcoili^s Is as big a mistake as can possibly be made. Otl 

^ the one hand. If the Raven i^ia trices Test Is thought of as a test of "Intel- 
"^llgence", one would not expect changes In such scores ^ a consequence of 
Follow-Through. On the other Hjand, we ^o have data that highly effective 
strategies for solving Raven type problems can indeed by taught to learners, 
causing dramatic changes in the scores that they earn on such tests" (Feuer- 
steJ^rr979, 1980).' If that is the case, then a large part of, what the 
Raven measures must be achievement . Therefore it would be Inexcusable, in 
the absence of such an explicit achievement goal, for a given Follow-Through 
mo^el to be evaluated u&j^li)zing this achievement test, which has no demon- 
strated content validity for any programs which are being assessed. It^ 
se^ms to be clear that the basic Follow-Through , evaluation required major 
compromise. It was compromised in the direction of setting up the study to 
fit resources, instrumentation, and unintended types of field implementa** 
tions^ By the time all those compromises had been made, very little could 
be said from the evaluation about a "Follow-Through program". If all Fol- 
low-Through programs were to be evaluated by the same narrow and, in my 
• opinipn, minimally relevant, criteria," program operatorj^ should at the very- 
least have had advance warning.. They should have beeii provided with the 

instruments so that they might adjust their programs to coincide with such 
'^ externally imposed objectives. We should have learned of the necessity for 
the development or selection of appropriate valid instrmpents for evalua- 
tion. Too much i,s at stake to permit casual attention* to this matter. If 
it is important to ask either research or evaluation questions, it is ira- 
portant that equal concern be reflected for the provision of measuries appro- 
priate to the task. It is not enough to "clean out the cupboard" and use 
]ieftover tests, since it is not only programs but their clients who are e- 
valuated ultimately. ' » 
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What is not revealed . As one reads the materials from a variety of 
Follow-Through evaluations, one is struck by how cold, remote and irrelevant 
to actual program ^operation the Results of the^e evaluations appear to be* ^ 
Because of the constraints of evaluation procedures, the result^ yield little 

m - ' ' 

that will help readers to .grs^sp a real flavor of what Follow-Through is all 
about. While important at one level, the debate between evaluators and their 
critics resembles more of a scholastic exercise than a vital function for edu- 
cational programs. One is left with the sinking feeling that no matter^who 
might win in the debates over units of analysis or the uise of analysis of co- 
variants as an appropriate analytical tool, little would have been learned 
from the pas^: evaluations to reveal the dynamic workings of the successful edu- 

cati^tmal process itself St basic positive results which it might indeed have 

, •» 
achieved. Therefore, in once sense, the lesson of Follqw-Through ,is more a 

lesson about what to do and what not to do in program evaluation than it is 
about how to do Follow-Through education or whether Follow-Through education 
worked. ' ' ' ■ 

I^rom what has been^ learned from the operation and evaluation of Follow- 

V 

Through programs to date, it is not possible to offer a bold redesign of 



education for. children who are -se^fea^^y the program. We are faced with an 
uncomfortable situation. On the surface, it would appear that we can do just 
as miich for Follow-Through children- by leav4.ng them completely alone as by 
involving them in special programs. As indicated earlier, X am unable to ac- 
'""^ cept such a conclusion for several reasons:' 

1, Follow-Through "programs" have yet to be described in terms whidi 
are appropriat;e for rigorous evaluation. 
' 2. Content-valid instrumentation for all Follow-Through program eval- 
uation does not exist at this time.. 

ERIC • , . • ^2 



3. There is a need to be clear about th6 rationale for a Follow-Through 
program. Is Follow- Through needed as an experiment to determine how to 

9 

solve the problem of how to educatie poor children? Or, is Follow- ^ 
Through to be an attempt to provide- the\ resources so that children who 

/ ' ' 

are now deprived of -Aormal school experiences can be given them? 

4. ilhere are many demonstrations which proVe the effectiveness of good 

' ... • • 
teaching. • , ^ . 

To date, only the* population that is or could be eligible for Follow- 
Thijugh has been a part of evaluation efforts. Secondly, even^in those cases, 
there has been insufficient documentation to provide a description of the 
actual versus tlje promised treatment of Fibllow-Through children among the 
many models or between the educational treatment of those who are provided ' • 
a Follow-Through treatment and Jthose comparisdn groups who were not a part of 
the treatment. But most .important of all, there is no documentation to com- 
pare the actual treatment which is offered to Follow-Through children and . 
children in non-Follow-Throu^h programs where children experience high rates 
of ^success in school achievement. 

As indicated earlier, the methodology, for making precision comparisons 
among educational treatments already exists. Jane Stallings at Sanford Re- 
search Institute, J. Brophy at the University of Texas, David Berliner, for- 

r 

marly of the Far West Regional Laboratory for Educational Research and Db^el- 

r 

opment, among others, have developed observational methods for classrooms. 
Such methods can be used both to verify the implementation of promised pr^- \^ 
grams, and to determine the nature of such distinctions as may exist among 
programs^ and between programs and regular good educational offerings. Up to 
now, the emphasis of evaluation^ has been on child outcomes • In the future , 
in the interest of val.id evaluation and in fairness to Follow-Through children , 
both cTiild outcomes and'school treatment must be considered simultaneously. 
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Th^Natlonal Institute of Education Is In an excellent position to struc- 
ture a View research and evaluation effort. In viaw of existing Follow-Through 
evaluation data, it is urgent that such an effort be initiated^and that it be 
comprehensive^ The low expectations for school achievement whicjji nLny educa- 
tors seem to have for .children like those who are served (jby Follow- Through pro- 
grams a,re fed by researc;h and evaluation programs which are underfunded and* . 

V 

therefore incomplete and invalid, ^limited in conce{ltion and therefore incom- 

♦ » 

pleta and inval'id'i or restricted to Follow- Through children and therefore in-^ 
complete and invalid. 

The Basic Lessons' of Follow-Through ^ ' 

There are a number things which can and should become lessons for educa- 
tors from the Fol^low^Through experience. They are as follows,' ^ • * 

Stop confusing research and evaluation . When the Office of Education asks 
the question, "Does Follow-Through work?" it sounds as If it is an evaluation 
question* In reality, it is not. It is a research question, for the simple 

reason that, at the outset, neither Follow-Through itself nor the Planned Var- 

.* * * 

iations were programs of known instructional quality. Evaluation implies the 

implementdfion of knowns. Neither, at the outset, were there testing and .evfil- 

nation instruments which were designed specifically for the purpose of evalu- . 

ating a full range o|^ Follow-Through programmatic outcomes; indeed, the state 

of the art in testing and measurement is such that the very4 process of design-* 

ing valid instruments may well be as large as the Follow-Through operation in 

♦ 

tt)tal. Further, the general goals of Follow-Through werp not then, nor are 
they now, restricted, and articulated- in a way that would allow a precision 
evaluation to proceed. In this sense, Follow-Through is not unlika "regular 
educatioA", In. fact, until the social programs of the 60's^were Introduced, 
it was, rare that educators were expected to prove that education "worked, as 
a price for fiscal support. 
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• Clarify Follov-Thtough purposes » At a public policy level,, it is neces- 
sary* to determine clearly if Follow-Through is to be a support or an experi- 
mental program. Whichever Is to be the case, the policy position must be 
supported with the best possible p^rofessional rationale. That rationale ^ 
should be grounded In' actual educational expetieuce and not in futuristic 
epeculation. In other words, if Follow-Through is to be support for a program 
of known quality, examples of successfully operating programs which serve as 
ipodels for public policy must be available. If the Follow-Through program is 
to be seen as a research program, 'then it is most unfair to Follow- Through 
children to implement it on a nati}.onal scale. ^ * 

Free Follow-Through from the^tate of the present art in evaluation . The 
plain truth is that professional ;judgment may be a better way of determining 
the va'lue of/ the Followr Through experience for children than the mechanism of 
traditional program evaluation. The state of the art in instrumentation for 
program evaluation is not Sufficiently developed to allow the future of Fol- 
low-Through prograite to ride' on traditional program evaluation outcomes, es- 
pecially since the state of the art in "educational programming is not so well 
developed that a unifom Follow- Through experience can be mandated at a na- 
tional level, nor shou^Ld it. Decisidns about the support of Follow-Through 
must be made long before the state of the art in evaluation can be improved 
substantially. ^Perh. . a combinat:ion of client satisfaction and professional 
judgment is the best that we can hope for on the evaluation side at the present 
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Build both a resear-ch and -evaluatio n capacity > ,My concerns and criticisms 

* 

with existing evaluation and research* are not intended in any way to pjiggest 
that either evaluation or research in educational programs such as Follow- 
Through should be eliminated. Quite to the contrary, both evaluation and re- 
search have the potential for making major contributions to the fi^d of educa- 
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tion. However, this cannot happen unless sufficient fiscal support is offered 
for that purpose. The demands and expectations of sponsors of research and 
evaluation must be jnet by an appropriate fiscal level, but above 4II, educa- 
tional researchers" must he given the time to do the job. The needs of public 

,1 

policy, makers for adequate information may be strong, yet educators must not 
be asked to do wlifl*f cannot be done.. ^ 

I ^ 

li^oncluslon * ^ * ^ 

It has been hard for many educator to realize* that superior school • 
achievement is associated with superior teaching. The belief persists among ^ 
many professionals and among influential members of the lay public that* fami- 
lies are pore responsible for school achievement thatn *are schools, overlook- ♦ 
ing the fact that families who have the resources aire able to guarantee access 
to the best schools for their children. There is little empirical research ^to 
document this phenomenon. However^ if takes little observati\)n in schpols 
which serve the children of the wealthy and in schools which serve children of 
the poor to see the dramatic differenced between them in educational experiences 
which they provide for children. The victims of the pervasive belief that 
"school experience" has minimal effect on children are -the children Who are 
served by Follow-Through programs. It is the continuing failure of educators 
to distinguish amoijg the variety of qualities of .school experiences which al- 
lows "school experience*'<,to be perceived as a common thing. It is this which 
allows the failure of educational systems to be seen as iailures of children 
and families. 

Follow-Through has not 'been a failure- I believe that' we may say that it 
has succeeded in direct proportion to its provision of appropriate resources 
to support good teaching- Follow-Through children have not failed to respond 
to good teaching. It remains for educational leaders and policy makers to 
stop the drift in thinking which leads to a loss of faith in children and edu- 

26 



cators. When we become clear about the goals of education^ and when we commit 

\ 

ou selves and our resources fully to those goals, the acra^mlc success for 
children will be assured. The alternative is a dangerous academic rationaliza 
tion and legitimation of the status quo. 
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